Charles  river  analytics 


Charles  River  Analytics  Inc. 
625  Mount  Auburn  St. 
Cambridge,  MA  02138 


+  p:  617.491.3474 
•  f:  617.868.0780 


www.cra.com 


15  MAR  2009 


Reference:  Government  Contract  No.  N00014-09-C-0050,  “Enhancing  Simulation- 

based  Training  Adversary  Tactics  via  Evolution  (ESTATE)” 

Charles  River  Analytics  Contract  No.  C08098 

Subject:  Contractor’s  Status  Report:  Quarterly  Status  Report  #1 

Reporting  Dates:  12/15/2008  -  03/15/2009 

Dear  Dr.  Hawkins, 

The  following  is  the  Contractor’s  Quarterly  Status  Report  for  the  subject  contract  for 
the  indicated  period.  During  this  reporting  period  work  has  concentrated  on  Task  1 : 
Identify  Training  Goals,  Task  4:  Develop  Trainee  Model  Processing,  and  Task  6:  Perfonn 
Simulation-based  Training  System  Integration 

1.  Summary  of  Progress 

1.1  Kickoff  Briefing 

A  kickoff  briefing  was  held  on  2  FEB  2009  at  the  Office  of  Naval  Research  in 
Arlington,  VA.  Present  were  Brad  Rosenberg  (Principal  Investigator)  from  Charles  River 
Analytics  and  Prof.  Jordan  Pollack  (Technical  Advisor)  from  Brandeis  University.  From 
the  Office  of  Naval  Research,  Dr.  Harold  Hawkins  (Program  Officer)  and  Ms.  Annetta 
Burger  were  present.  Brad  Rosenberg  led  the  discussion,  which  explored  the  program 
motivation,  approach,  and  goals  for  the  base  year  period. 

During  the  discussion,  it  was  revealed  that  the  ESTATE  effort  falls  within  a  larger 
initiative  for  “Training  Adaptability”  within  the  Office  of  Naval  Research.  The  adaptive 
training  research  initiative  can  be  decomposed  into  two  fields:  1)  creating  adaptive 
training  systems,  and  2)  creating  systems  that  train  adaptability.  ESTATE  primarily 
addresses  the  first  field,  with  the  added  benefit  of  indirectly  addressing  the  second. 
ESTATE  supplies  adversaries  in  a  simulated  environment  that  adapt  to  the  strengths  and 
weaknesses  of  an  individual  trainee,  creating  a  tailored  training  experience  to  facilitate 
accelerated  skills  development  and  sustainment.  This  promotes  active  learning,  reduces 
costs,  and  adds  realism  to  training  systems.  Furthermore,  by  providing  an  adaptive 
adversary,  ESTATE  compels  a  trainee  to  adapt,  training  measure-countermeasure 
problem-solving  skills. 

The  initial  research  under  previous  efforts  for  ESTATE  was  primarily  focused  in  the 
convoy  operations  domain.  While  that  domain  was  determined  to  be  high-priority  given 
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the  current  operational  climate,  it  was  not  a  good  match  for  training  opportunities  that  an 
adaptive  simulation-based  training  system  could  provide.  During  the  kickoff  briefing,  it 
was  decided  that  the  target  domain  should  be  reopened,  with  a  focus  towards  squad-level 
behaviors.  We  are  currently  exploring  training  literature  to  determine  current  gaps  in 
training  at  this  level  and  its  match  to  a  simulation-based  training  approach. 

Discussion  also  led  to  programmatics  regarding  when  decisions  need  to  made 
regarding  continuing  the  research  under  the  Year  2  option  (Option  1).  As  that  decision 
needs  to  be  made  by  OCT/NOV,  it  would  best  serve  for  a  demonstration  or  briefing  be 
prepared  for  late  summer,  early  fall  such  as  the  SEP  2009  timeframe.  We  have  adapted 
our  program  plan  accordingly  to  have  a  demonstration  ready  by  this  time. 

Furthermore,  at  the  time  of  the  kickoff  briefing,  the  ESTATE  program  had  only 
partial  funding  allocated  for  the  base  year  portion  of  the  effort.  As  of  18  FEB  2009,  the 
base  year  period  has  now  been  fully  funded  and  Option  1  has  been  exercised. 

It  was  also  requested  during  the  kickoff  briefing  that  a  Quad  Chart  be  generated  that 
is  simpler,  conveys  the  overall  program  goals,  and  could  be  used  as  a  promotional 
introduction  to  the  ESTATE  effort  for  a  military  officer.  A  copy  of  that  Quad  Chart  is 
presented  below. 
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1.2  Requirements  Analysis 

During  the  indicated  period,  we  performed  a  literature  review  in  the  area  of  learning 
science  to  help  ground  ESTATE  system  design  and  development  in  solid  concepts. 
Additionally,  we  engaged  with  PM  TRASYS  to  help  determine  training  requirements  for 
the  Marine  Corps. 

1.2.1  Litera  ture  Review 

Several  learning  theories  help  provide  a  framework  for  training  systems.  These 
learning  theories  fall  under  three  categories:  behaviorism,  cognitivism,  and 
constructivism.  Behaviorism  is  concerned  with  how  the  environment  shapes  behavior 
and  focuses  learning  on  the  acquisition  of  new  behavior  through  conditioning  (classical 
condition  or  operant  condition).  Cognitivism  attempts  to  look  beyond  overt  behavior  and 
focuses  instead  on  memory  and  prior  knowledge  as  the  two  critical  aspects  of  learning. 
Alternatively,  constructivism  views  learning  as  a  process  on  the  active  construction  of 
knowledge  based  on  current  and  past  knowledge.  It  is  this  last  category,  constructivism, 
that  we  are  concerned  with.  Aspects  of  constructivism  are  found  in  experiential  learning 
and  situated  cognition.  The  premise  of  these  concepts  is  “learning  by  doing”,  which  has 
strong  parallels  to  the  Marine  Corps  training  approach  of  “train  as  you  fight.”  As  a  result, 
simulation-based  training  enables  Marines  to  “train  as  you  fight,”  and  is  the  approach 
taken  within  ESTATE. 

During  the  indicated  period,  we  discovered  that  the  ESTATE  framework  is 
functionally  similar  to  the  field  of  intelligent  tutoring  systems.  Intelligent  tutoring 
systems  derived  from  computer-aided  instruction  in  education  and  rose  in  popularity 
during  the  1980s- 1990s  given  increases  in  computational  power.  Intelligent  tutoring 
systems  are  comprised  of  three  major  components:  an  expert  model,  a  student  model,  and 
an  instructor  model.  The  expert  model  represents  subject  matter  expertise  and  problem 
solving  ability  and  is  used  to  help  compare  a  learner  with  an  expert.  The  student  model 
attempts  to  evaluate  the  learner’s  performance  to  detennine  knowledge,  skills,  and 
abilities.  Finally,  the  instructor  model  encodes  instructional  methods  that  are  appropriate 
to  the  target  domain  (represented  in  the  expert  model)  and  the  learner.  Within  ESTATE, 
the  expert  model  is  encoded  as  the  mission  essential  task  lists  and  key  perfonnance 
indicators  as  specified  by  training  doctrine.  The  student  model  is  the  trainee  model 
processing  component  of  ESTATE.  The  instructional  model  is  ESTATE’S  adversary 
adaptation  component.  We  intend  to  leverage  this  knowledge  as  we  move  forward  with 
the  ESTATE  research. 

In  “Foundations  of  Intelligent  Tutoring  Systems”  ,  VanLehn  describes  student 
modeling  as  inferring  a  model  of  the  student’s  current  understanding  of  the  subject 
material.  This  process  is  referred  to  as  diagnosis.  The  goal  of  diagnosis  is  to  uncover 
hidden  cognitive  states  of  the  student  from  observable  behavior.  These  hidden  cognitive 
states  are  called  latent  traits  in  the  literature,  and  refer  to  the  knowledge,  skills,  and 


1  VanLehn,  K.  (1988).  Student  Modeling.  In  Poison,  M.C.,  Richardson,  J.J.  (Ed.),  Foundations  of 
Intelligent  Tutoring  Systems  (pp.  55-78),  Hillsdale,  NJ:  Lawrence  Erlbaum  Associates  Inc. 
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abilities  of  our  trainees  in  ESTATE.  The  student  model  can  then  be  used  for  many 
different  things  within  an  intelligent  tutoring  system,  one  of  which  is  problem  generation. 
This  is  the  current  usage  within  the  ESTATE  system,  generating  an  appropriate  challenge 
that  promotes  learning. 

There  are  several  dimensions  to  diagnosis,  including  bandwidth,  target  knowledge 
type,  and  student-expert  difference.  Bandwidth  is  the  amount  of  information  that  can  be 
gleaned  from  the  system  for  diagnosis.  It  includes  approximate  mental  states, 
intermediate  states,  and  final  states.  Approximate  mental  states  provide  transparency  into 
the  cognitive  processes  of  the  student  and  provide  the  highest  amount  of  bandwidth  to 
glean  information  from.  Intermediate  states,  however,  are  closed  to  what  is  going  inside 
the  head  of  the  trainee  and  only  observes  behavior  (e.g.,  mid-game  board  positions  in 
chess).  Final  states  only  enable  the  capture  of  information  at  the  end  of  the  interaction 
(e.g.,  win/loss,  final  score),  and  provide  the  lowest  amount  of  bandwidth.  While  more 
information  usually  beneficial,  it  adds  complexity  to  capture  and  utilize  that  data  within 
the  intelligent  tutoring  system.  For  the  purposes  of  ESTATE,  we  will  assume  that  we  will 
begin  with  the  final  states  (i.e.,  the  performance  of  the  trainee  given  a  challenge)  with  an 
eye  towards  intermediate  states  in  the  future. 

Target  knowledge  types  distinguish  how  to  apply  knowledge  in  the  student  model  to 
the  problem.  VanLehn  describes  two  types  of  knowledge,  procedural  knowledge  and 
declarative  knowledge.  Procedural  knowledge  makes  decisions  based  on  local  knowledge 
by  applying  procedures  based  on  the  situation  and  applying  it  to  a  task  while  declarative 
knowledge  searches  the  whole  knowledge  base.  Under  ESTATE,  we  will  assume  that  the 
knowledge  we  are  trying  to  convey  is  procedural,  in  the  context  of  performing  a  specific 
function  or  task,  which  is  learned  experientially. 

Finally,  the  last  dimension  of  interest  is  student-expert  difference.  This  feature 
attempts  to  define  how  a  model  of  the  student’s  knowledge  differs  from  an  expert.  The 
simplest  form  is  of  overlay,  where  the  student  is  an  expert  with  some  missing  knowledge. 
However,  this  can  be  made  more  complex  by  imagining  that  a  student  additionally  has 
some  misconceptions,  or  “bugs”,  that  hinder  learning.  In  this  format,  a  bug  library  can  be 
created  that  defines  the  incorrect  knowledge  the  student  has.  If  these  bugs  can  be 
decomposed  into  their  component  parts,  then  a  bug  part  library  can  be  created,  where 
bugs  can  be  assembled  dynamically  that  are  specific  to  an  individual  student.  For  the 
purposes  of  ESTATE,  we  will  make  the  assumption  of  overlay,  that  a  student  is  an  expert 
that  is  simply  missing  certain  knowledge,  skills,  or  abilities. 

Using  intelligent  tutoring  systems  research  as  a  guiding  principle  in  the  ESTATE 
framework,  we  then  attempted  to  formulate  an  initial  training  system  within  the  scope  of 
the  base  year  period.  While  a  real-time  military  simulation  is  the  eventual  goal,  a  more 
feasible  approach  is  needed  to  design,  develop,  and  evaluate  an  adaptive  training  system. 
Since  we  have  made  the  initial  assumptions  of  using  final  states,  teaching  procedural 
knowledge,  and  using  an  overlay  form  of  student  modeling,  we  have  targeted  challenge  / 
response  games  as  a  first  approach  for  ESTATE.  Within  a  challenge  /  response 
framework,  a  challenge  problem  is  presented  to  the  trainee,  who  then  attempts  to  respond 
to  the  challenge  in  the  appropriate  manner,  doing  so  either  correctly  or  incorrectly  (a 
graded  scale  of  performance  is  also  possible).  Based  on  this  response,  a  new  challenge 

DISTRUBITION  A.  Approved  forpublic  release;  distribution  isunlimited 


4 


can  be  generated  and  presented  next  that  promotes  learning.  We  believe  this  provides  a 
general  framework  within  scope.  It  is  also  possible  to  envision  that  a  challenge  /  response 
game  can  be  conceived  of  as  a  generalization  of  a  tactical  simulation,  where  the  challenge 
is  a  tactical  situation  with  simulated  entities  and  the  response  of  the  trainee  is  realized  as 
specific  actions  in  that  situation.  The  figure  below  represents  the  ESTATE  architecture  in 
this  new  conceptual  framework. 


On-line  Execution 


Problems 


Under  the  upcoming  period,  we  plan  on  adapting  some  of  the  challenge  /  response 
games  from  the  BeeWeb  program  (http://www.beeweb.org)  from  Brandeis  University  to 
this  framework.  Specifically,  we  are  looking  into  MoneyBee,  a  money  exchange  game 
that  teaches  algebra,  logic,  and  estimation.  We  are  hoping  this  simple  game  will  allows  us 
to  help  define  problem  difficulty  dimensions,  trainee  skill  dimensions,  and  a  dynamic 
difficulty  assessment  and  problem  generation  system.  We  are  also  looking  into  how  item 
response  theory2  may  have  a  role  in  this  process. 

1.2.2  Discussion  with  PM  TRASYS 

During  the  indicated  period,  initial  discussions  were  held  with  Ms.  Nancy  Harmon  of 
PM  TRASYS  starting  3  MAR  2009  to  help  identify  training  requirements  and  gaps  (both 
from  a  domain  and  technical  perspective)  and  to  gain  information  about  constructive 


Baker,  F.B.,  (2001),  Basics  of  Item  Response  Theory,  ERIC  Clearinghouse  on  Assessment  and 
Evaluation,  College  Park,  MD. 
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training  systems  at  PM  TRASYS.  Ms.  Hannon  formerly  led  PM  TRASYS  in  constructive 
simulations,  but  is  now  focused  on  immersive  simulation  technologies. 

Regarding  training  requirements  and  gaps,  Ms.  Harmon  indicated  that  she  is  in 
possession  of  Marine  Corps  training  material  that  includes  task  lists  and  performance 
measurement  criteria  that  is  releasable  to  help  guide  ESTATE  system  development.  We 
are  currently  in  the  processing  of  obtaining  that  material.  Additionally,  Ms.  Harmon 
indicated  she  has  connection  to  military  subject  matter  experts  (i.e.  Captain,  Gunnery 
Sergeant)  currently  aiding  PM  TRASYS  in  this  area  that  could  help  in  the  requirements 
analysis  process.  These  same  SMEs  could  then  participate  in  formative  and  summative 
evaluations  of  the  ESTATE  software  prototype.  The  goals  of  the  ESTATE  program 
would  then  be  to  enter  into  a  Memorandum  of  Agreement  (MO  A)  or  Technology 
Transfer  Agreement  (TTA)  with  PM  TRASYS. 

In  addition  to  training  requirements  and  gaps,  the  discussion  with  Ms.  Harmon  helped 
assess  simulation  as  a  training  system  technology.  It  was  noted  that  while  the  Deployable 
Virtual  Training  Environment  (DVTE)  is  being  utilized  in  PM  TRASYS,  that  the 
Combined  Arms  Network  (CAN)  component  is  receiving  less  use  than  the  tactical 
decision-making  simulations  (TDS)  within  DVTE.  Specifically,  VBS2  is  receiving 
considerable  focus.  VBS2  is  being  looked  at  as  an  interim  technology  for  creating 
immersive  simulated  environments  with  an  eye  towards  the  DARPA  REAL  WORLD 
program  to  supplant  it  in  the  future.  As  a  result,  we  will  focus  our  initial  assessment  on 
VBS2  as  a  target  simulation-based  training  system.  Ms.  Harmon  is  currently  putting  us  in 
touch  with  the  DVTE  Project  Officer  so  we  can  gain  access  to  DVTE. 

2.  Scheduled  Items 

•  Obtain  training  manuals  and  other  guiding  documents  from  PM  TRASYS  to  help 
determine  target  training  domains 

•  Obtain  DVTE  from  the  program  office  for  in-house  evaluation  and  development 

•  Primary  focus  on  VBS2  to  assess  integration  points  for  intelligent  behavior  of 
simulated  entities. 

•  Design  of  a  formal  framework  for  challenge  /  response  games  that  is  extensible  for 
initial  system  development  and  experimentation 

•  Application  of  the  MoneyBee  game  to  the  challenge  /  response  game  framework  for 
experimentation 

Sincerely, 


Brad  Rosenberg 
Principal  Investigator 
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